Skip to content

Add visualization tool for topology cli #369

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
azreenz wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Add visualization tool for topology cli #369

azreenz wants to merge 7 commits into from

Conversation

@azreenz
Copy link
Collaborator

@azreenz azreenz commented Jun 3, 2025

Add visualization tool for topology cli to create ASCII art to visualize block and tree slurm topologies

  • Add --generate_visualization option (may want to change how this is named)
  • Add --visual_block_size option to determine what each block is supposed to look like (default as 18 to mimic a rack)
  • Add visualize function to Topology class
  • run function now returns the topology string so visualize function can take it in to generate the visualization
  • If nodes are less than max_block_size then we fill out with 'X'
  • Commented out blocks are still visualized but include a "** This block is ineligible for scheduling because # of nodes < min block size**" line in the header
  • Add test cases to validate new function

Example Block topology visual:

block 1  : # of Nodes = 18
ClusterUUID + CliqueID : 5e797fc6-0f46-421a-8724-0e102c0f723c
|-----------------|-----------------|-----------------|
| hpcbench-hpc-1  | hpcbench-hpc-5  | hpcbench-hpc-9  |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-13 | hpcbench-hpc-17 | hpcbench-hpc-21 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-25 | hpcbench-hpc-29 | hpcbench-hpc-33 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-37 | hpcbench-hpc-41 | hpcbench-hpc-45 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-49 | hpcbench-hpc-53 | hpcbench-hpc-57 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-61 | hpcbench-hpc-65 | hpcbench-hpc-69 |
|-----------------|-----------------|-----------------|

block 2  : # of Nodes = 18
ClusterUUID + CliqueID : 5e797fc6-0f46-421a-8724-0e102c0f721e
|-----------------|-----------------|-----------------|
| hpcbench-hpc-2  | hpcbench-hpc-6  | hpcbench-hpc-10 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-14 | hpcbench-hpc-18 | hpcbench-hpc-22 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-26 | hpcbench-hpc-30 | hpcbench-hpc-34 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-38 | hpcbench-hpc-42 | hpcbench-hpc-46 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-50 | hpcbench-hpc-54 | hpcbench-hpc-58 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-62 | hpcbench-hpc-66 | hpcbench-hpc-70 |
|-----------------|-----------------|-----------------|

block 3  : # of Nodes = 18
ClusterUUID + CliqueID : 5e797fc6-0f46-421a-8724-0e102c0f724b
|-----------------|-----------------|-----------------|
| hpcbench-hpc-3  | hpcbench-hpc-7  | hpcbench-hpc-11 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-15 | hpcbench-hpc-19 | hpcbench-hpc-23 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-27 | hpcbench-hpc-31 | hpcbench-hpc-35 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-39 | hpcbench-hpc-43 | hpcbench-hpc-47 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-51 | hpcbench-hpc-55 | hpcbench-hpc-59 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-63 | hpcbench-hpc-67 | hpcbench-hpc-71 |
|-----------------|-----------------|-----------------|

block 4  : # of Nodes = 18
ClusterUUID + CliqueID : 5e797fc6-0f46-421a-8724-0e102c0f722a
|-----------------|-----------------|-----------------|
| hpcbench-hpc-4  | hpcbench-hpc-8  | hpcbench-hpc-12 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-16 | hpcbench-hpc-20 | hpcbench-hpc-24 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-28 | hpcbench-hpc-32 | hpcbench-hpc-36 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-40 | hpcbench-hpc-44 | hpcbench-hpc-48 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-52 | hpcbench-hpc-56 | hpcbench-hpc-60 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-64 | hpcbench-hpc-68 | hpcbench-hpc-72 |
|-----------------|-----------------|-----------------|

Example Block topology visualization with nodes less than max_block_size

block 1  : # of Nodes = 6
ClusterUUID + CliqueID : N/A N/A
|---------------|---------------|---------------|
| vis0603-gpu-2 | vis0603-gpu-5 | vis0603-gpu-1 |
|---------------|---------------|---------------|
| vis0603-gpu-3 | vis0603-gpu-4 | vis0603-gpu-6 |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|

Example Block Topology visual with block with nodes less than min block size

block 1  : # of Nodes = 6
ClusterUUID + CliqueID : N/A N/A
** This block is ineligible for scheduling because # of nodes < min block size 8**
|---------------|---------------|---------------|
| vis0603-gpu-4 | vis0603-gpu-2 | vis0603-gpu-5 |
|---------------|---------------|---------------|
| vis0603-gpu-1 | vis0603-gpu-6 | vis0603-gpu-3 |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|

Example Tree topology visual:

Switch 5 (root)
├── Switch 0 (3 nodes)
│   ├── hpcbench-hpc-36
│   ├── hpcbench-hpc-39
│   └── hpcbench-hpc-42
├── Switch 1 (6 nodes)
│   ├── hpcbench-hpc-1
│   ├── hpcbench-hpc-35
│   ├── hpcbench-hpc-38
│   ├── hpcbench-hpc-41
│   ├── hpcbench-hpc-44
│   └── hpcbench-hpc-49
├── Switch 2 (3 nodes)
│   ├── hpcbench-hpc-37
│   ├── hpcbench-hpc-45
│   └── hpcbench-hpc-46
├── Switch 3 (2 nodes)
│   ├── hpcbench-hpc-40
│   └── hpcbench-hpc-43
└── Switch 4 (2 nodes)
    ├── hpcbench-hpc-47
    └── hpcbench-hpc-48

@azreenz azreenz requested review from aditigaur4 and bwatrous June 4, 2025 16:20
@azreenz azreenz marked this pull request as ready for review June 4, 2025 16:20
bwatrous
bwatrous reviewed Jun 4, 2025
View reviewed changes
bwatrous
bwatrous reviewed Jun 4, 2025
View reviewed changes
bwatrous
bwatrous reviewed Jun 4, 2025
View reviewed changes
@azreenz azreenz force-pushed the azreenzaman/topology-visualization branch 2 times, most recently from 4a212bf to 3f08f03 Compare June 4, 2025 18:56
bwatrous
bwatrous previously approved these changes Jun 4, 2025
View reviewed changes
aditigaur4
aditigaur4 reviewed Jun 4, 2025
View reviewed changes
azure-slurm/slurmcc/topology.py
str: ASCII visualization of the topology.
"""

if self.topo_type == TopologyType.BLOCK:
Copy link
Collaborator

@aditigaur4 aditigaur4 Jun 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you have extremely big if else blocks-- its always best to condense them into functions.

if self.topo_type == TopologyType.BLOCK:
    visualize_block_topology()
else: 
    # assuming tree
    visualize_tree_topology()

Copy link
Collaborator

@aditigaur4 aditigaur4 Jun 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would also be nice if visualize_topology* functions actually get moved to util since its not really part of core topology logic. You also dont really need any data from the topology class. Just the string and the block size. Thats an optional suggestion though.

@azreenz azreenz force-pushed the azreenzaman/topology-visualization branch from 3f08f03 to 14d5ba0 Compare June 30, 2025 15:17
@azreenz azreenz force-pushed the azreenzaman/topology-visualization branch from 14d5ba0 to 8d018cb Compare June 30, 2025 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@aditigaur4 aditigaur4 aditigaur4 left review comments

@bwatrous bwatrous bwatrous left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@azreenz @bwatrous @aditigaur4